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object image moves a distance in the direction and at a second speed proportional to the first 
speed with which the touching contact touches the object. : „ 

REMARKS 

PENDING CLAIMS 

In accordance with the election of the Group I claims and the cancellation of the 
remaining claims of Groups II, III and IV in the prior response, there are now pending herein 
claims 1 1-48, 55-60, 71-76, 83-91 and 101-102, as correctly listed in the Action. The withdrawn 
and cancelled claims are also correctly listed. 



REJECTED CLAIMS: CLAIMS 11-15, 19-48, 55-60, 71-76, 83-91 AND 101-102 
OBJECTED-TO CLAIMS: CLAIMS 16-18 
ITEM 3: DOUBLE PATENTING REJECTION 

All of the pending claims, as listed above, are rejected for obviousness-type double 
patenting, which the Action notes may be overcome by a timely filed terminal disclaimer . 

A suitable Terminal Disclaimer is filed concurrently herewith and in accordance with 
which it is respectfully requested that the double patenting rejection be withdrawn. 

ITEM 4: OBJECTION TO DRAWINGS UNDER 37 CFR 1.83(a) RE CLAIM RECITATION OF 
"ELASTIC" 

The Examiner objects to the drawings under 37 CFR 1 .83(a) for not showing the claimed 
feature of "elastic." More particularly, the Examiner cites the recitation of "elastic" as found in 
claims 39 and 42, wherein "object information" specifies a "manipulation type of 'elastic'" (per 
claim 39, claim 42 being the same but without the internal quotation marks surrounding the 
word). Applicants assume the Examiner would similarly object to the recitation in claim 23 of 
"object information specifying the displayed object to be of an elastic type." 

It is respectfully submitted that the Examiner errs in his objections to the drawings. 
Particularly, the "distort-restore manipulation" illustrated in Figs. 11(a) and 11(b), discussed 
beginning at page 13, line 10 of the specification, relates to "an elastic object... pressed by a 
finger on the display surface of the display unit 3." 
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Fig. 1 1(b) furthermore is characterized as a "display information table..." which "defines 
the object type as 'elastic', meaning that the object can be distorted and restored according to a 
pressure applied thereon by a finger and as shown in Fig. 1 1(a)...." Fig. 1 1(b). in fact, includes a 
specific example of "object type" as being "elastic." 

A recitation of "the object type as 'elastic'." appears directly in originally filed claim 10/1 , 
at line 21. 

Hence, it is respectfully submitted that the drawings do support the use of the term 
"elastic" in the claims since the specification characterizes specific, illustrated objects of the 
drawings as having the characteristic of "elastic." 

Note also that claim 1 1 . from which claim 23 depends, includes a generic recitation of 
"object information for the displayed object image, including data relative to a type of the 
displayed object." Claim 23 thus recites a specific such "type of the displayed object" as "an 
elastic type" - and an "elastic" type of object is clearly definite, consistent with the dictionary 
definition of "elastic," viz.: 

capable of recovering size and shape after deformation 

Claims 39 and 42 depend respectively from independent claims 37 and 40 which 
likewise recite storage of object information: "each object information specifying a responsive 
manipulation type of each object" and thus generically encompass the "elastic" manipulation 
type of the dependent claims 39 and 42. 

The Examiner did not object to the drawings based on the generic "object type" 
recitations of the independent claims, which supports applicants' contention that the objections 
to the drawings, relative to the specific "elastic type" recitations of the dependent claims, are 
misplaced. 

ITEMS 5 AND 6: REJECTION OF CLAIM 18 UNDER 35 USC § 112, H2, FOR 
INDEFINITENESS 

The rejection is seemingly based on the Examiner's miscomprehension of a rather 
ordinary English language construction in the claim 18 recitations. Particularly, the touching 
contact is "coming in a direction toward the object" when it touches the object. Nevertheless, 
claim 18 is amended in the foregoing to change "coming" to -after moving-. 
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The Examiner's question as to - "what coming object is the applicant referring to?" -- is 
respectfully submitted to be irrelevant, since reversing the order of the words "coming object" 
from that of the claim language. 

The Examinees further criticism of the phrase "at a speed higher than a predetermined 
speed" is misplaced (1 ) since the Examiner has omitted the operative recitation of the clause 
and (2) by the emphasis in bold, which distorts the original claim language. More particularly, 
claim 18 recites a method step which is performed "in response to the detected characteristics... 
wherein the omitted operative function is that of "controlling the display of the object so that the 
object image moves...: 

(1) a distance in the direction and... 

(2) at a speed proportional to the speed with which the touching contact .touches the 
object." 

To further clarify the recitation in question, applicants have further amended same to 
recite the " second speed proportional to the first speed with which the touching contact touches 
the object." 

The Examiner could consider, as an example, an embodiment of the method of the 
invention in which a human foot is displayed, moving in a direction toward a soccer ball, from a 
position displaced from the soccer ball and at a (first) speed higher than a predetermined speed 
when the foot kicks (i.e., makes "touching contact with") the soccer ball. In accordance with the 
invention, the method controls the display of the soccer ball (i.e., the displayed object image) 
such that it moves in the same direction in which the foot was moving at the time the foot kicked 
the soccer ball and at a (second) speed proportional to the (first) speed with which the foot 
kicked the soccer ball. 

Accordingly, it is submitted that claim 18 satisfies the standards of definiteness of § 1 12, 
2 and at the rejection of same should be withdrawn. 

ITEMS 7-13: PRIOR ART REJECTIONS 

Herein, all of the pending claims as above identified are rejected, claim 44 for 
anticipation by Ohuchi and the remainder for obviousness over combinations of seven different 
references of which Levine et al., Santo et al., Ohuchi and Dunthorn are newly cited and relied 
upon in this Action. However, Murasaki, USP 5,867,158 having a U.S. filing date of August 30, 
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1996 and Hagiuda, USP 5,953,008 having a U.S. filing date of October 1, 1997 and citing 
benefit of a provisional application filed March 27, 1997, are not references against the present 
application which, instead, claims parent benefit to application Serial No. 08/437,563 filed May 9, 
1995, a continuation of U.S. Serial No. 07/953,101 filed September 29, 1992 and which, 
moreover, enjoys the benefit of an even earlier effective filing date based on a claim of foreign 
priority benefit to JP 2-258232 filed October 7, 1991. 

For convenience of reference, the rejections based on the remaining references which 
are effective as prior art against the present application are summarized in the following table. 



35 USC § 102(b) Rejection 



Claim 


Reference 


44 


Ohuchi (USP 4,903,012) 


35 USC § 103(a) Rejections 


Claims 


References 


11, 15, 23, 34, 36, 37, 39, 48, 55-60, 101 and 
102 


Levine (USP 5,060, 1 35), Sato (USP 
4,947,156) & Ohuchi (USP 4,903,012) 


12, 35, 71 and 76 


Levine (USP 5,060,135), Sato (SP 4,947,156) 
& Dunthorn (USP 4,914,624) 


10-22, 38,41 


Levine (USP 5,060,135), Sato (USP 
4,947,156) & Charwan (EPO 179147) 


13, 14, 83-90 


Levine (USP 5,060,135) & Sato (USP 
4,947,156) 


91, 100 


Levine (USP 5,060,135), Sato (USP 
4,947,156, Ohuchi (USP 4,903,012) & 
Takahashi (USP 4,954,967) 



The foregoing rejections are respectfully traversed. 
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Ohuchi 

In item 8, claim 44 is rejected for anticipation under 35 U.S.C. § 102(b) by Ohuchi. The 
rejection is respectfully traversed. 

Ohuchi relates only to solving a prior art problem of responding correctly to a finger touch 
relative to a displayed cursor and to improving the ability to move a cursor in a mouse mode in 
response to a finger touch on the screen. The system displays a test cursor on a video display 
1 and detects a finger 5 which is put on the test cursor 4 to effect a coordinate system input 
operation. (Col. 5, lines 12-15) A correction amount then is generated corresponding to a 
deviation between the position entered from the finger or other touching element and the test 
cursor. (Col. 6, lines 55-62) 

Ohuchi is devoid of any of the features of the invention as set forth in claim 44 relating to 
the detecting a position of touching contact and any changes thereof relative to an object and 
has absolutely no teaching of "an object type which specifies physical properties of the object 
and position information which specifies a position of the displayed object image." 

Levine et al 

Levine et al. is the primary reference relied upon in support of each of the six 103(a) 
rejections. For the reasons set forth below, Levine et al. is submitted to be inadequate and 
incapable of supporting the aforesaid six 1 03 rejections whether taken singularly or in 
combination with any of the secondary references, and in light of which all of the rejections are 
respectfully traversed. 

Levine et al. discloses system displays of memos or folders, etc. on a display monitor. 
The system has a tablet capable of sensing contact of a stylus and wherein the tablet is 
provided separate from the display. The user can move the folders or memo on the display 
monitor by operation of the tablet. More particularly, as described in the abstract and beginning 
at col. 8, the computer device simulates a desk referred to as the "user system desk" and user 
interaction with items on the desk. In relation to Fig. 1A, stylus 14 may be used on an upper 
planar surface of a tablet 16 to perform certain tasks, such as repositioning displayed items or 
selecting a displayed item for further processing. (Col. 9, lines 1-5) More particularly, the driven 
stylus and response thereto are said to provide four different manners of operation, or four 
different effects, two being "touch and lift" and "touch and move." "Touch and lift" is evoked only 
if the stylus end is removed from the tablet surface within a specific time interval (e.g., one 
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second) or else the operation is not invoked. "Touch and move" requires maintaining the stylus 
in contact with the tablet surface and then moving the stylus, while maintaining the contact, for 
more than a preset number of pixels, for example, four pixels. The third and fourth manners of 
operation are not relevant. The system also provides icons which can provide different 
operations, based on being touched by the stylus. (Col. 8, lines 23-37) Stamp representations 
of documents are similar to icons and are subject to movement by the stylus operation. (Col. 1 1 , 
lines 28-46) 

While col. 5, line 61 - col. 6, line 5 of Levine is cited as disclosing "object manipulation 
simulation", it is respectfully submitted this is not what is disclosed. Instead, Levine merely 
describes each of various accessories being represented by "a respective illustrative 
icon. ..(which). ..the user selects. ..by a touch and lift method of use...." In other words, Levine 
does not interpret the nature of the touching contact but, instead, different accessories are 
selected by merely a common "touch and lift" action of a stylus which is unrelated to the 
feature of the present invention of interpreting the nature of the touching contact and using same 
to define the intended simulated manipulation of the object image. Further, col. 3, lines 32-41 of 
Levine et al. does not relate to any "recognized, simulated manipulation of the object" but again 
deals with "representations of documents... which are preferably miniaturized or reduced images 
of one page of the document." (Col. 3, lines 32-34) 

It is respectfully submitted that the reference fails to teach touching contact which 
"simulates a manipulation of the object" and which outputs "touch information corresponding to 
the sensed touching contact and changes of the touching contact" and which, moreover, 
provides for detecting from that information a "corresponding object manipulation simulated 
thereby. 

Further, there is no teaching of changing the display of the object image "in accordance 
with the recognized, simulated manipulation of the object and object information for the 
displayed object image, including data relative to a type of the displayed object." (See claim 1 1 ) 
Further, Levine et al. not only lack any teaching of the generic "type of the displayed object" of 
the independent claim 1 1 , it lacks, as well, all of the various different, specific "detected 
characteristics" of the touching contact and of "object types" of the dependent claims. 

Claim 25, furthermore, recites "object information, including an object type, which 
specifies physical properties of the object" and, moreover, the combination of "object type which 
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specifies physical properties of the object and", as does claims 27 and 29. Claim 32 recites the 
touching contact as "simulating an actual touching contact with an actual object...." 

Clearly, Levine et al. lack any suggestion of responding to the sensed touching contact 
relative to a displayed image for "simulating a manipulation of the object" and thereby to 
recognize therefrom "the corresponding object manipulation simulated thereby" and the function 
of a producing a display of the object image in accordance with the "recognized, simulated 
manipulation of the object and object information for the displayed object image including data 
relative to a type of the displayed object." (See also claims 34, 36 and 37). Claim 40 includes a 
similar recitation of the object information "specifying a responsive manipulation type of the 
object" and pursuant to which the display of the object image is changed in accordance with the 
sensed touch information and at least one of plural object information...." These features, in 
various combinations and permutations, are set forth as well by the subsequent independent 
claims 43 through claim 48. 

Claims 55-60 and 71-74 set forth a controller and/or apparatus for use with a display 
device to manipulate an object displayed on the display device, consistent with the foregoing 
features and claims 75-102 set forth various methods, apparatus and/or computer readable 
memory storing a computer program, affording such simulated manipulation of a displayed 
object, all thereof patentably distinguishing over the disclosure of Levine et al. 

In accordance with the foregoing, it is submitted to be clear that all of the § 103 rejections 
rely on Levine and, in view of the deficiencies thereof, the rejections should be withdrawn since 
Levine fails to teach or even suggest the character of simulated manipulation of a displayed 
image in accordance with the present invention. 

It furthermore is submitted that the combinations of Levine et al. with the secondary 
references are unsupported by any demonstration of prima facie obviousness and for that 
reason do not overcome the basis deficiency of Levine et al. Moreover, these references, 
individually, lack any relevant teaching in relation to the present invention. 

Sato et al. 

Sato et al., USP 4,947,156, (cited as "Santo et al." in the Action) is cited only for the 
proposition of teaching the display of a manipulated image of a table as in Levine et al. on an 
LCD or EL type display. 
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Sato merely discloses an LCD apparatus having a transparent touch panel mounted on 
the viewing surface of the LCD, used to write characters by a stylus. The apparatus moves a 
display location of the written characters so that writer can see them easily. The Sato reference 
thus is unrelated to simulating a manipulation of an object as disclosed and claimed herein and 
cannot overcome the deficiency of Levine et al. 

Levine et al. discloses a system in which a display monitor is separated from a touch 
sensor, as the Examiner concedes. The Examiner insists, however, that this will allow the user 
to save space by integrating the writing table and the display. He presumably intends to say that 
it would have been obvious, to one skilled in the art at the time of the invention was made, to 
have been motivated to use Sato et al.'s method with the Levine et al's because of that reason. 
But Sato et al.'s method relates to writing characters and moving positions of the written 
characters and as to which the technical field and object of invention are different from Levine's. 
Accordingly, there is no prima facie obviousness supporting the proposed combination of the 
two references. 

It is submitted that the Sato et al. teaching is irrelevant to the invention that is claimed 
herein and moreover fails to overcome the deficiencies of Levine. Sato et al. simply deal with a 
problem of parallax when writing on any type of such table. There is no basis for the Examiner 
to assert that the deficiency of parallax addressed by Sato et al. exists in Levinee et al. or would 
be suitable for implementation in Levine et al. 

Dunthorn 

Dunthorn is relied upon in item 9 of the Action in combination with Levine and Sato et al. 

Dunthorn teaches sensing a second contact on a touch screen thereby to generate a 
virtual push button — in Figure 1, the initial contact is the forefinger at 22 and the second contact 
is the thumb at 29 and, when both are present, the virtual push button is established at location 
26. (Col. 6, lines 48-55) There is absolutely no relationship between the two touch contact 
purpose of Dunthorn and the "two touching contacts ... located at opposite sides of the object 
image ..." recited in claim 12/1 1. inter alia . As noted, "26" is not an "object image" but a "virtual 
button" created when the two spaced contacts 22 and 29 occur simultaneously. 

Ohuchi 

In item 1 0 of the Office Action, the Examiner refers to the center of gravity of an object 
with regard to dependent claims 17, 59 and 60. He cites Ohuchi, but admits that it does not 
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teach a center of gravity. As he further admits, the cited prior art calculates a distance between 
the location of the displayed cursor and of the touching contact on the display monitor, but does 
not teach that displayed object is moved. Therefore, there is no apparent basis to combine 
Levine and Ohuchi. 

Charwat (EP 0179147) 

Item 1 1 relies on the combination with Levine et al., of Sato et al., Charwat, the latter for 
teaching displaying an image in a rolling condition. 

Clearly, Charwat does not overcome the deficiencies of Levine et al. Moreover, the 
Action is silent as to the combination of the three references which are relied upon — not even 
mentioning Sato et al.-- and hence is fatally deficient. 

Conclusion 

The art of record neither renders obvious nor even suggests the invention defined by the 
pending claims. 

Accordingly, there being no other objections or rejections, it is submitted that the 
application is in condition for allowance, which Action is earnestly solicited. 

If there are any additional fees associated with filing of this Amendment, please charge 
the same to our Deposit Account No. 19-3935. 



Respectfully submitted, 



STAAS & HALSEY LLP 



Date: March 25, 2002 




H. J. Sta&fe 

Registration No. 22,010 



700 Eleventh Street, NW, Suite 500 
Washington, D.C. 20001 
(202) 434-1500 



CERTIFICATE UNDER 37 CFR 1.8(a) 
I hereby certify that this correspondence is being de- 
posited with the United States Postal Service as first 
class mail in an envelope addressed to: Commissioner 
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VERSION WITH MARKINGS TO SHOW CHANGES MADE 

IN THE CLAIMS: 

Please AMEND the following claims: 

1 1 . (AS UNAMENDED) A method for simulating manipulation of an object utilizing a 
displayed image of the object, said method comprising the steps of: 

sensing touching contact relative to the displayed image of the object, the touching 
contact simulating a manipulation of the object, and outputting touch information corresponding 
to the sensed touching contact and changes of the touching contact; 

detecting, from the output touch information, characteristics of said touching contact 
including the selected location on the object image of the touching contact and changes of the 
touching contact and recognizing therefrom the corresponding object manipulation simulated 
thereby; and 

changing the display of the object image in accordance with the recognized, simulated 
manipulation of the object and object information for the displayed object image, including data 
relative to a type of the displayed object. 

12. (AS UNAMENDED) A method according to claim 1 1 , wherein: 

in response to detected characteristics indicating that two touching contacts are located 
at opposite sides of the object image and move and stop with a distance therebetween, 
controlling the display of the object image on the display device so that the object moves on a 
display surface of the display device from where the two touching contacts are located at 
opposite sides of the object image to where the two touching contacts stop with the distance 
therebetween. 

1 3. (AS UNAMENDED) A method according to claim 1 1 , wherein: 

in response to the detected characteristics indicating that the touching contact position is 
located at the object and moves while keeping in contact with the object and the object is 
sufficiently large relative to, and extends beyond, the display or only a portion of the object 
image is displayed on the display device, controlling the display of the object by scrolling the 
object image while displaying same. 



11 



Serial No.: 09/419,516 



14. (AS UNAMENDED) A method according to claim 13, wherein: 

in response to the detected characteristics indicating that a moving, touching contact 
stops moving, controlling the display of the object image on the display device so that the 
scrolling of the object image stops on the display. 

15. (AS UNAMENDED) A method as recited in claim 11, further comprising storing 
information specifying a position of a portion of the object image being displayed on the display 
device, relative to the whole object image. 

16. (AS UNAMENDED) A method according to claim 1 1 , wherein: 

in response to the detected characteristics indicating that the touching contact is located 
at a geometric center or at a center of gravity of the object and moves and stops while 
maintaining contact with the object, controlling the display of the object image so that the object 
image moves from where the touching contact is located at the geometric center or the center of 
gravity to where the moving, touching contact stops moving. 

17. (AS UNAMENDED) A method according to claim 1 1 , wherein: 

in response to the detected characteristics indicating that the touching contact is located 
at a position off a geometric center or off a center of gravity of the object and moves and stops 
while maintaining contact with the object, controlling the display of the object image so that the 
object image moves while rotating, from a position where the touching contact begins to a 
position where the touching contact stops. 

18. (ONCE AMENDED) A method according to claim 11, wherein: 

in response to the detected characteristics indicating that the touching contact touches 
the object [coming] after moving in a direction toward the object, from a position apart therefrom 
and at a first speed higher than a predetermined speed, controlling the display of the object so 
that the object image moves a distance in the direction and at a second speed proportional to 
the first speed with which the touching contact touches the object. 

19. (AS UNAMENDED) A method according to claim 11, further comprising: 

in response to the object information specifying the displayed object to be of a rollable 
type, displaying the object image in a rolling condition. 
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20. (AS UNAMENDED) A method according to claim 19, further comprising: 

in response to the detected characteristics indicating that the touching contact touches 
the object image, moving and stopping while maintaining contact with the object, displaying the 
object image in a rolling condition. 

21. (AS UNAMENDED) A method according to claim 19, further comprising: 

in response to the detected characteristics indicating that the touching contact moves on 
the object while maintaining said touching contact with the object, displaying the object image in 
a rolling condition and changing a positional relationship between the object image and the 
touching contact on the object image in accordance with the movement of the touching contact 
on the object. 

22. (AS UNAMENDED) A method according to claim 19, further comprising: 
displaying the object image in a rolling condition so as to move the object image in a 

direction of the movement of the touching contact on the object. 

23. (AS UNAMENDED) A method according to claim 1 1 , further comprising: 
displaying the object image in a state of distortion and/or restoration, according to the 

object information specifying the displayed object to be of an elastic type. 

24. (AS UNAMENDED) A method according to claim 1 1 , further comprising: 
sensing an amount of pressure applied to the object image and, when the detected 

characteristic of the touching contact exceeds a specific amount of pressure, displaying the 
object image in a degree of distortion and/or restoration according to the amount of, and 
changes in, pressure of the touching contact. 

25. (AS UNAMENDED) An image display and manipulating method, comprising: 
displaying an image of an object; 

sensing touching contact relative to the displayed object image and outputting 
corresponding touch information; 
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detecting, from the received touch information, characteristics of the touching contact, 
the characteristics including the position on the object image of the input touching contact and 
changes of the position of the input'touching contact; and 

changing the display of the object images in accordance with the characteristics and 
object information including an object type, which specifies physical properties of the object. 

26. (AS UNAMENDED) A method according to claim 25, further comprising 
detecting changes of the position of the input touching contact. 

27. (AS UNAMENDED) A method of manipulating a displayed image of an object, 
comprising: 

displaying an image of the object; 

detecting a position of touching contact relative to the object image and changes of the 
touching contact; and 

changing the display of the object image in accordance with the detected characteristics 
and object information including at least an object type which specifies physical properties of the 
object and position information which specifies a position of the displayed object image. 

28. (AS UNAMENDED) A method according to claim 27, further comprising 
detecting changes of the touching contact. 

29. (AS UNAMENDED) A method of manipulating a displayed image of an object, 
comprising: 

sensing characteristics of a touching contact relative to the displayed image of the 
object, the characteristics including at least a position of the touching contact and changes 
therein, and outputting corresponding touch information representing a simulated movement of 
the object responsive to the characteristics of the touching contact; and 

recognizing a type of manipulation of the object image in accordance with the information 
representing a simulated movement of the object and an object type specifying physical 
properties of the object and, correspondingly, manipulating and displaying the object image. 

30. (AS UNAMENDED) A method according to claim 29, further comprising sensing 
changes in the position of the touching contact. 
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31 . (AS UNAMENDED) A method as recited in claim 30, further comprising storing 
information specifying a position of a portion of the object being displayed, relative to the whole 
object. 

32. (AS UNAMENDED) A display method comprising: 
producing a display of an image of an object; 

sensing touching contact relative to the displayed object image, simulating an actual 
touching contact wittran actual object corresponding to the displayed image of the object; 

detecting characteristics of the touching contact including a position of the touching 
contact relatively to the object image and changes of the position of the touching contact; and 

changing the display of the object in accordance with the object information and the 
detected characteristics. 

33. (AS UNAMENDED) A method according to claim 32, further comprising 
detecting changes of the position of the touching contact. 

34. (AS UNAMENDED) A computer readable medium storing therein a computer 
program affording simulated manipulation of an object in accordance with an image of the object 
displayed on a display surface of a display device, said computer program comprising: 

a first function of responding to sensed touching contact relative to the displayed image 
of the object, simulating a manipulation of the object, and to changes in the touching contact for 
producing and outputting corresponding touch information; 

a second function of detecting, from the output touch information, characteristics of said 
touching contact including the selected location on the object image of the touching contact and 
changes of the touching contact and recognizing therefrom the corresponding object 
manipulation simulated thereby; and 

a third function of changing the display of the object image in accordance with the 
detected characteristics of the touching contact, the recognized, simulated manipulation of the 
object and object information for the displayed object image including data relative to a type of 
the displayed object. 
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35. (AS UNAMENDED) A computer readable medium according to claim 34, 
wherein, in response to said second function detecting characteristics indicating that two 
touching contacts are located at opposite sides of the object image and, further, move and stop 
with a distance therebetween, said third function changes the display of the object image so that 
the object moves on a display surface of the display device from where the two touching 
contacts are located at opposite sides of the object image to where the two touching contacts 
stop with the distance therebetween. 

36. (AS UNAMENDED) A computer program product for controlling a computer, the 
program product comprising: 

a recording medium readable by the computer; 

means responsive to touching contact, and to changes in the touching contact, relative to 
an object displayed on a display surface of a display device, simulating manipulation of the 
object, for producing and outputting touch information corresponding to the sensed touching 
contact and representing the simulated manipulation of the object; 

first subroutine means, responsive to the output touch information, for detecting the 
characteristics of the touching contact and recognizing therefrom the corresponding object 
manipulation simulated thereby; and 

second subroutine means for changing the display of the object image in accordance 
with the detected characteristics of the touching contact, the recognized, simulated manipulation 
of the object and object information for the displayed object image including data relative to a 
type of the displayed object image. 

37. (AS UNAMENDED) An apparatus affording simulated manipulation of an object in 
accordance with an image of the object display surface of a displayed on a display device, 
comprising: 

a display device displaying an image of an object; 

an input device responsive to a touching contact relative to the displayed object image, 
the touching contact simulating a manipulation of the object, and outputting corresponding touch 
information; 

a storage unit storing a plurality of object information, each object information specifying 
a responsive manipulation type of each object; and 
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a display controller changing the display of the object image in accordance with the 
touch information and the stored object information specifying a manipulation type of the object. 

38. (AS UNAMENDED) An apparatus according to claim 37, wherein: 

said storage unit stores an object information specifying that the manipulation type is 
,, rollable ,, ; and 

in response to the touch information and said object information, said display controller 
displays the object image in a rolling condition. 

39. (AS UNAMENDED) An apparatus according to claim 37, wherein: 

said storage unit stores an object information specifying a manipulation type of "elastic"; 

and 

in response to the touch information and said object information, the display controller 
displays the object image in a distortion and/or restoration condition. 

40. (AS UNAMENDED) A method for simulating manipulation of an object using a 
displayed image of the object, comprising: 

displaying an image of an object; 

responding to a touching contact relative to the displayed object image, the touching 
contact simulating a manipulation of the object, and outputting corresponding touch information; 
and 

changing the display of the object image in accordance with the touch information and at 
least one of plural object information, each object information specifying a responsive 
manipulation type of the object. 

41. (AS UNAMENDED) A method according to claim 40, wherein: 

said one of plural object information specifies that the manipulation type is "rollable"; and 
comprises 

in response to the touch information and said object information, displaying the object 
image in a rolling condition. 

42. (AS UNAMENDED) A method according to claim 40, wherein said one of plural 
object information specifies that the manipulation type is elastic, further comprising: 
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in response to the touch information and said object information, displaying the object 
image respectively in a distorted or a restored condition. 

43. (AS UNAMENDED) A storage medium storing a process displaying an image of 
an object by sensing touching contact relative to the displayed object image and outputting 
corresponding touch information; detecting, from the received touch information, characteristics 
of the touching contact, the characteristics including the position on the object image of the input 
touching contact and any changes of the position of the input touching contact; and changing 
the display of the object image in accordance with the characteristics and an object information 
including an object type, which specifies physical properties of the object. 

44. (AS UNAMENDED) A storage medium storing a process displaying an image of 
the object by detecting a position of touching contact relative to the object image and any 
changes of the touching contact and changing the display of the object image in accordance 
with the detected characteristics and object information including at least an object type which 
specifies physical properties of the object and position information which specifies a position of 
thedisplayed object image. 

45. (AS UNAMENDED) A storage medium storing a process displaying an image of an 
object by sensing characteristics of a touching contact relative to the displayed image of the 
object, the characteristics including at least a position of the touching contact and any changes 
therein, and outputting corresponding touch information representing a simulated movement of 
the object responsive to the characteristics of the touching contact; and recognizing a type of 
manipulation of the object image in accordance with the information representing a simulated 
movement of the object and an object type specifying physical properties of the object and, 
correspondingly, manipulating and displaying the object image. 

46. (AS UNAMENDED) A storage medium as recited in claim 45, storing information 
specifying a position of a portion of the object being displayed, relative to the whole object. 

47. (AS UNAMENDED) A storage medium producing a display of an image of an 
object by sensing touching contact relative to the displayed object image, simulating an actual 
touching contact with an actual object corresponding to the displayed image of the object; 
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detecting characteristics of the touching contact including a position of the touching contact 
relatively to the object image and any changes of the position of the touching contact; and 
changing the display of the object in accordance with the object information and the detected 
characteristics. 

48. (AS UNAMENDED) A computer readable medium storing therein a computer 
program affording simulated manipulation of an object using a displayed image of the object, 
said computer program comprising: 

a first function of responding to a touching contact relative to the displayed object image, 
the touching contact simulating a manipulation of the object, and outputting corresponding touch 
information; and 

a second function of changing the display of the object image in accordance with the 
touch information and at least one of plural object information, each said object information 
specifying a responsive manipulation type of the corresponding object. 

55. (AS ONCE AMENDED) A controller for use with a display device to manipulate 
an object displayed on the display device and an input device responsive to a touching contact 
relative to the displayed object and simulating said manipulation, the input device producing 
corresponding touch information and the controller comprising: 

a detector detecting, from the touch information, characteristics of said touching contact 
including the selected location on the displayed object of the touching contact and changes of 
the touching contact and recognizing therefrom the corresponding object manipulation simulated 
thereby; and 

a display controller controlling to change the display of the object in accordance with the 
recognized, simulated manipulation of the object and object information for the displayed object, 
including data relative to a type of the displayed object 

56. (AS ONCE AMENDED) A controller for use with a display device to manipulate 
an object displayed on the display device and an input device responsive to a touching contact 
relative to the displayed object, the touching contact simulating said manipulation and the input 
device producing corresponding touch information; the controller comprising: 

a storage unit storing a plurality of object information, each object information specifying 
a responsive manipulation type of each object; and 
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a display controller controlling to change the display of the object in accordance with the 
touch information and the stored object information specifying a manipulation type of the object. 

57. (AS ONCE AMENDED) An apparatus for use with a display device to manipulate 
an object displayed on the display device, the apparatus comprising: 

an input device responding to a manipulation to the displayed object and producing 
corresponding input information; 

a storage unit storing object information specifying an object and its center of gravity; 

a display controller controlling to change the display of the object in accordance with the 
input information and the object information. 

58. (AS ONCE AMENDED) An apparatus according to claim 57, wherein: 

said input device is responsive to a touching contact relative to the displayed object , the 
touching contact simulating said manipulation, and produces corresponding touch information; 
and 

said display controller controls the display of the object in accordance with the touch 
information and the object information. 

59. (AS ONCE AMENDED) An apparatus according to claim 57, wherein, in 
response to touch information indicating a movement such that the touching contact touches the 
object at its center or its center of gravity, moves and stops while keeping in contact with the 
object, said display controller recognizes the movement as a push manipulation and controls the 
display of the object so that the object moves from where the touching touches the object to 
where the touching stops. 

60. (AS ONCE AMENDED) An apparatus according to claim 57, wherein in 
response to touch information indicating a movement such that the touching contact touches 
said object at a position off the center or the center of gravity thereof, moves and stops on said 
touch panel while keeping in contact with the object, said display controller controls the object on 
the display device so that the object moves while rotating from where the touching contact 
touches the object to where the touching contact stops. 
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71 . (AS ONCE AMENDED) An apparatus for use with a display device to manipulate 
an object displayed on the display device, the apparatus comprising: 

an input device inputting a manipulation to the displayed object and producing 
corresponding input information; 

a display controller controlling to change the display of the object on the display device 
so that the object moves on the display device from where two touches touch respective, 
opposite sides of the object to where the two touches stop, in accordance with the input 
information indicating a movement such that the two touches touch respective, opposite sides 
of the object and move. 

72. (AS ONCE AMENDED) An apparatus according to claim 71 , wherein said input 
device is responsive to the two touches relative to the displayed object and simulating said 
manipulation, and outputs corresponding said input information ; and 

the display controller controlling to change the display of the object on the display device 
so that the displayed object moves on the display device from where the two touches touch 
respective, opposite sides of the displayed object to where the two touches stop touching the 
object, in accordance with the touch information indicating a movement such that the two 
touches touch the respective, opposite sides of the object and move. 

73. (AS ONCE AMENDED) An apparatus of claim 72, further comprising: 

a storage unit storing position information which specifies the position where the object is 
displayed on the display device; 

wherein the display controller, in accordance with the touch information and display 
position information, recognizes a movement such that the two touches touch the respective, 
opposite sides of the object and move, and controls the object on the display device so that the 
object moves on the display device from where the two touching touch both sides of the object 
to where the two touching stop. 

74. (AS ONCE AMENDED) A controller for use with a display device to manipulate 
an object displayed on the display device and an input device inputting a manipulation to the 
displayed object and producing corresponding input information, the controller comprising: 

a display controller controlling to change the display of the object on the display device 
so that the object moves on the display device from where the two touches touch respective, 
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opposite sides of the object to where the two touches stop such touching, in accordance with the 
input information indicating a movement such that the two touches touch the respective, 
opposite sides of the object and move. 

75. (AS UNAMENDED) A method for simulating manipulation of a displayed object, 
comprising: 

displaying an object; 

responding to a manipulation to the displayed object at an input device which produces 
corresponding input information; 

controlling to change the display of the object on the display device so that the object 
moves on the display device from where the two touches touch the respective, opposite sides of 
the object to where the two touches stop such touching, in accordance with the input information 
indicating a movement such that the two touches touch the respective, opposite sides of the 
object and move. 

76. (AS ONCE AMENDED) A computer readable medium storing therein a computer 
program affording simulated manipulation of displayed object, said computer program 
comprising: 

a first function of responding to a manipulation to the displayed object at an input device 
which produces corresponding input information; 

a second function of controlling to change the display of the object on the display device 
so that the object moves on the display device from where the two touches touch the respective, 
opposite sides of the object to where the two touches stop such touching, in accordance with the 
input information indicating a movement such that the two touches touch the respective, 
opposite sides of the object and move. 

83. (AS ONCE AMENDED) An apparatus for use with a display device to manipulate 
an object displayed on the display device, the apparatus comprising: 

an input device responsive to a touching contact, relative to the displayed object, and 
producing corresponding input information; 

a storage unit storing object information about a large object extending beyond a display 
area of the display device; and 
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a display controller, in accordance with the object information and the input information 
indicating a movement such that the touching contact moves a distance more than a 
predetermined distance or the touching contact moves at a speed higher than a predetermined 
speed, controlling to display the object on the display device in a scroll condition. 

84. (AS ONCE AMENDED) An apparatus according to claim 83, wherein said 
display controller controls to start the scroll condition at a first speed and gradually decreases 
the scroll condition to a second, lower speed. 

85. (AS ONCE AMENDED) An apparatus according to claim 84, wherein said first 
speed of the scroll condition depends on said speed at which, or said distance at once which, 
said touching contact moves. 

86. (AS ONCE AMENDED) An apparatus according to claim 83, wherein said 
display controller controls to decrease the speed of the scroll condition in a case where another 
touching contact does not occur. 

87. (AS ONCE AMENDED) An apparatus according to claim 84, wherein said 
display controller controls to continue the scroll condition in a case where another touching 
contact occurs before the scroll condition steps. 

88. (AS TWICE AMENDED) An apparatus according to claim 83, wherein said 
display controller controls to stop the scroll condition in a case where the touching contact stops 
in the scroll condition of the displayed object. 

89. (AS TWICE AMENDED) A controller for use with a display device to manipulate 
an object displayed on the display device and an input device responsive to a touching contact 
relative to the displayed object and producing corresponding input information, the controller 
comprising: 

a storage unit storing object information about a large object extending beyond a display 
area of the display device; and 

a display controller, in accordance with the object information and the input information 
indicating a movement, controlling to display the object on the display device in a scroll 
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condition, such that the touching contact moves a distance more than a predetermined distance 
or the touching contact moves at a speed higher than a predetermined speed. 

90. (AS TWICE AMENDED) A method for simulating manipulation of a displayed 
object, comprising: 

displaying a part of an object which extends beyond a display area; 

responding to a touching contact to the displayed object, at an input device responsive to 
a touching contact to the displayed object and producing corresponding input information; and 

controlling to display the object on the display device in a scroll condition, in accordance 
with the input information indicating a movement such that the touching contact moves a 
distance more than a predetermined distance or the touching contact moves at a speed higher 
than a predetermined speed. 

91 . (AS TWICE AMENDED) A computer readable medium storing therein a 
computer program affording simulated manipulation of displayed object, said computer program 
comprising: 

a first function of responding a manipulation to a displayed object from an input device 
responsive to a touching contact relative to the displayed object and producing corresponding 
input information, said displayed object extending beyond a display area; and 

a second function of controlling to display the object on the display device in scroll 
condition, in accordance with the input information indicating a movement such that the touching 
contact moves a distance more than a predetermined distance or the touching contact moves at 
a speed higher than a predetermined speed. 

101 . (AS UNAMENDED) An apparatus controlling a display of an object in 
accordance with a simulated manipulation of the object produced by a corresponding touching 
contact on an input device, relative to the displayed object, the input device producing 
corresponding touch information and the apparatus comprising: 

a detector detecting, from the touch information, characteristics of said touching contact 
including the selected location on the displayed object of the touching contact and changes of 
the touching contact and recognizing therefrom the corresponding object manipulation simulated 
thereby; and 
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a display controller controlling to change the display of the object in accordance with the 
recognized, simulated manipulation of the object and object information for the displayed object, 
including data relative to a type of the displayed object. 

102. (AS UNAMENDED) An apparatus controlling a display of an object in 
accordance with a simulated, selected manipulation of the object produced by corresponding 
touching contact on an input device, relative to the displayed object, the input device outputting 
corresponding touch information and the apparatus comprising: 

a storage unit storing a plurality of object information, each object information specifying 
a responsive manipulation type of each object; and 

a display controller controlling to change the display of the object in accordance with the 
touch information and stored object information specifying a manipulation type of the object. 
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